An Algebra for the Control of Stochastic Systems: Exercises in Linear Algebra

A new algebraic framework is introduced based on stochastic matrices. Together with several new operators, the set of stochastic matrices is shown to constitute a vector space, an inner-product space, and an associative algebra. The new zero vector is the uniform probability distribution and linear addition is akin to statistical independence. This new stochastic algebra allows Markov chains and control problems to be reexamined in the familiar constructs of a vector space. A stochastic calculus furthermore allows Markov control problems to be linearized thus creating a connection to the classic linear time-invariant control theory.